Individual differences in representational gesture production are associated with cognitive and empathy skills.

Substantial individual variation exists in the frequency of gestures produced while speaking. This study investigated the associations between cognitive abilities, empathy levels, and personality traits with the frequency of representational gestures. A cartoon narration task and a social dilemma solving task were used to elicit gestures. Predictor variables were selected based on prior research on individual differences in gesture production and the cognitive and communicative functions of gestures in speech. Our findings revealed that an increased frequency of representational gestures was associated with higher empathy levels in the cartoon narration task. However, in the social dilemma solving task, a higher frequency of representational gestures was associated with lower visuospatial working memory, spatial transformation, and inhibition control abilities. Moreover, no significant relationships were found between verbal working memory, personality traits, and the frequency of representational gestures in either task. These findings suggested that predictor variables for representational gesture production vary depending on the nature of the gesture elicitation task (e.g., spatiomotoric vs. abstract topics). Future research should examine the relationship between individuals' cognitive abilities, empathy and gesture production with across a broader range of topics and in more ecologically valid contexts.

Subliminal perception of others' physical pain induces personal distress rather than empathic concern.

BACKGROUND: What is our immediate reaction when we witness someone experiencing pain? The empathy-altruism hypothesis predicts that observers would display empathy and a tendency to approach the person in pain. Alternatively, the threat value of pain hypothesis (TVPH) argues that others' pain serves as a signal of threat and should induce observers' avoidance response. METHODS: To examine these two hypotheses, three experiments were conducted. The experiments aimed to investigate the impact of subliminal exposure to others' physical pain on participants' emotional and behavioural responses. RESULTS: The results revealed that subliminal pain priming resulted in faster response and attentional bias to fearful faces compared to sad faces (Experiment 1), faster reaction times in recognizing fear-related words compared to anger-related words during a lexical decision task (Experiment 2), and faster avoidance responses towards anger-related words, as opposed to approaching responses towards positive words (Experiment 3). CONCLUSIONS: The consistent findings across all experiments revealed that subliminal perception of pain scenes elicited fear emotion and immediate avoidance responses. Therefore, the outcomes of our study provide supportive evidence for the TVPH.

Encoding and decoding hidden meanings in face-to-face communication: Understanding the role of verbal and nonverbal behaviors in indirect replies.

In everyday conversation, we often use indirect replies to save face of our interlocutor (e.g., "Your paper does have room for improvement"). Six experiments were conducted to examine the role of verbal and nonverbal behaviors in the production and comprehension of indirect replies. In Experiments 1a and 1b, participants engaged in question-answer exchanges designed to elicit four types of replies (i.e., direct, indirect, lie, and neutral). Results showed that uncertainty terms, discourse markers and head tilt were most uniquely associated with the production of indirect replies. In Experiments 2a, 2b, 3a and 3b, participants categorized the types of replies in video clips of real participants in Experiments 1a and 1b. Results showed that nonverbal behaviors enhanced the performance and boosted the confidence in the identification of indirect replies. Furthermore, uncertainty terms, discourse markers and head tilt were also the most reliable cues for identifying indirect replies. Finally, the extent to which people relied on verbal and nonverbal cues to identify an indirect reply was context dependent. The more informative the verbal/nonverbal information was, the fewer nonverbal/verbal cues that contributed to the identification of indirect replies. Our results demonstrated that people integrate verbal and nonverbal information to enhance their understanding of the intended meaning in indirect replies. Findings from the current research provide an initial step toward developing a comprehensive and unified model of the production and comprehension of indirect replies, which takes both verbal and nonverbal behaviors into account. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Intrinsic Cerebro-Cerebellar Functional Connectivity Reveals the Function of Cerebellum VI in Reading-Related Skills.

The engagement of the cerebellum VI in reading was reported in both typically developing and dyslexic readers. However, it is still not clear how the cerebellum VI contributes to reading. Here we have examined the correlation of intrinsic cerebro-cerebellar functional connectivity with two critical reading-related skills-phonological awareness (PA) and rapid automatized naming (RAN)-with fMRI technology. Specifically, we tested the hypothesis that the cerebellum may contribute to reading either by phonological skills or by automatizing skills. We chose the left and right cerebellum VI as ROIs, and we calculated the intrinsic cerebro-cerebellar functional connectivity during a resting state. We further explored whether and how cerebro-cerebellar resting state functional connectivity (RSFC) is associated with individuals' reading-related skills including PA and RAN. The results showed that the functional connectivity between the left supramarginal gyrus and bilateral cerebellum VI was related to RAN, and the connectivity between the left insula and right cerebellum VI was related to PA. However, the effect of PA did not survive after the RAN was regressed out. Control analyses further confirmed that it was the intrinsic cerebro-cerebellar functional connectivity rather than the local cerebellar functionality that associated with phonological awareness ability and rapid automatized naming ability. For the first time, the relationship between cerebro-cerebellar resting state functional connectivity and specific reading-related skills has been explored, and this has deepened our understanding of the way the cerebellum VI is involved in reading.

Graph theoretical analysis of functional network for comprehension of sign language.

Signed languages are natural human languages using the visual-motor modality. Previous neuroimaging studies based on univariate activation analysis show that a widely overlapped cortical network is recruited regardless whether the sign language is comprehended (for signers) or not (for non-signers). Here we move beyond previous studies by examining whether the functional connectivity profiles and the underlying organizational structure of the overlapped neural network may differ between signers and non-signers when watching sign language. Using graph theoretical analysis (GTA) and fMRI, we compared the large-scale functional network organization in hearing signers with non-signers during the observation of sentences in Chinese Sign Language. We found that signed sentences elicited highly similar cortical activations in the two groups of participants, with slightly larger responses within the left frontal and left temporal gyrus in signers than in non-signers. Crucially, further GTA revealed substantial group differences in the topologies of this activation network. Globally, the network engaged by signers showed higher local efficiency (t(24)=2.379, p=0.026), small-worldness (t(24)=2.604, p=0.016) and modularity (t(24)=3.513, p=0.002), and exhibited different modular structures, compared to the network engaged by non-signers. Locally, the left ventral pars opercularis served as a network hub in the signer group but not in the non-signer group. These findings suggest that, despite overlap in cortical activation, the neural substrates underlying sign language comprehension are distinguishable at the network level from those for the processing of gestural action.

How do gestures influence thinking and speaking? The gesture-for-conceptualization hypothesis.

People spontaneously produce gestures during speaking and thinking. The authors focus here on gestures that depict or indicate information related to the contents of concurrent speech or thought (i.e., representational gestures). Previous research indicates that such gestures have not only communicative functions, but also self-oriented cognitive functions. In this article, the authors propose a new theoretical framework, the gesture-for-conceptualization hypothesis, which explains the self-oriented functions of representational gestures. According to this framework, representational gestures affect cognitive processes in 4 main ways: gestures activate, manipulate, package, and explore spatio-motoric information for speaking and thinking. These four functions are shaped by gesture's ability to schematize information, that is, to focus on a small subset of available information that is potentially relevant to the task at hand. The framework is based on the assumption that gestures are generated from the same system that generates practical actions, such as object manipulation; however, gestures are distinct from practical actions in that they represent information. The framework provides a novel, parsimonious, and comprehensive account of the self-oriented functions of gestures. The authors discuss how the framework accounts for gestures that depict abstract or metaphoric content, and they consider implications for the relations between self-oriented and communicative functions of gestures. (PsycINFO Database Record

Beat that Word: How Listeners Integrate Beat Gesture and Focus in Multimodal Speech Discourse.

Communication is facilitated when listeners allocate their attention to important information (focus) in the message, a process called "information structure." Linguistic cues like the preceding context and pitch accent help listeners to identify focused information. In multimodal communication, relevant information can be emphasized by nonverbal cues like beat gestures, which represent rhythmic nonmeaningful hand movements. Recent studies have found that linguistic and nonverbal attention cues are integrated independently in single sentences. However, it is possible that these two cues interact when information is embedded in context, because context allows listeners to predict what information is important. In an ERP study, we tested this hypothesis and asked listeners to view videos capturing a dialogue. In the critical sentence, focused and nonfocused words were accompanied by beat gestures, grooming hand movements, or no gestures. ERP results showed that focused words are processed more attentively than nonfocused words as reflected in an N1 and P300 component. Hand movements also captured attention and elicited a P300 component. Importantly, beat gesture and focus interacted in a late time window of 600-900 msec relative to target word onset, giving rise to a late positivity when nonfocused words were accompanied by beat gestures. Our results show that listeners integrate beat gesture with the focus of the message and that integration costs arise when beat gesture falls on nonfocused information. This suggests that beat gestures fulfill a unique focusing function in multimodal discourse processing and that they have to be integrated with the information structure of the message.

Co-thought and co-speech gestures are generated by the same action generation process.

People spontaneously gesture when they speak (co-speech gestures) and when they solve problems silently (co-thought gestures). In this study, we first explored the relationship between these 2 types of gestures and found that individuals who produced co-thought gestures more frequently also produced co-speech gestures more frequently (Experiments 1 and 2). This suggests that the 2 types of gestures are generated from the same process. We then investigated whether both types of gestures can be generated from the representational use of the action generation process that also generates purposeful actions that have a direct physical impact on the world, such as manipulating an object or locomotion (the action generation hypothesis). To this end, we examined the effect of object affordances on the production of both types of gestures (Experiments 3 and 4). We found that individuals produced co-thought and co-speech gestures more often when the stimulus objects afforded action (objects with a smooth surface) than when they did not (objects with a spiky surface). These results support the action generation hypothesis for representational gestures. However, our findings are incompatible with the hypothesis that co-speech representational gestures are solely generated from the speech production process (the speech production hypothesis).

Electrophysiological and Kinematic Correlates of Communicative Intent in the Planning and Production of Pointing Gestures and Speech.

In everyday human communication, we often express our communicative intentions by manually pointing out referents in the material world around us to an addressee, often in tight synchronization with referential speech. This study investigated whether and how the kinematic form of index finger pointing gestures is shaped by the gesturer's communicative intentions and how this is modulated by the presence of concurrently produced speech. Furthermore, we explored the neural mechanisms underpinning the planning of communicative pointing gestures and speech. Two experiments were carried out in which participants pointed at referents for an addressee while the informativeness of their gestures and speech was varied. Kinematic and electrophysiological data were recorded online. It was found that participants prolonged the duration of the stroke and poststroke hold phase of their gesture to be more communicative, in particular when the gesture was carrying the main informational burden in their multimodal utterance. Frontal and P300 effects in the ERPs suggested the importance of intentional and modality-independent attentional mechanisms during the planning phase of informative pointing gestures. These findings contribute to a better understanding of the complex interplay between action, attention, intention, and language in the production of pointing gestures, a communicative act core to human interaction.

Synchronization of speech and gesture: evidence for interaction in action.

Language and action systems are highly interlinked. A critical piece of evidence is that speech and its accompanying gestures are tightly synchronized. Five experiments were conducted to test 2 hypotheses about the synchronization of speech and gesture. According to the interactive view, there is continuous information exchange between the gesture and speech systems, during both their planning and execution phases. According to the ballistic view, information exchange occurs only during the planning phases of gesture and speech, but the 2 systems become independent once their execution has been initiated. In all experiments, participants were required to point to and/or name a light that had just lit up. Virtual reality and motion tracking technologies were used to disrupt their gesture or speech execution. Participants delayed their speech onset when their gesture was disrupted. They did so even when their gesture was disrupted at its late phase and even when they received only the kinesthetic feedback of their gesture. Also, participants prolonged their gestures when their speech was disrupted. These findings support the interactive view and add new constraints on models of speech and gesture production.

Individual differences in frequency and saliency of speech-accompanying gestures: the role of cognitive abilities and empathy.

The present study concerns individual differences in gesture production. We used correlational and multiple regression analyses to examine the relationship between individuals' cognitive abilities and empathy levels and their gesture frequency and saliency. We chose predictor variables according to experimental evidence of the functions of gesture in speech production and communication. We examined 3 types of gestures: representational gestures, conduit gestures, and palm-revealing gestures. Higher frequency of representational gestures was related to poorer visual and spatial working memory, spatial transformation ability, and conceptualization ability; higher frequency of conduit gestures was related to poorer visual working memory, conceptualization ability, and higher levels of empathy; and higher frequency of palm-revealing gestures was related to higher levels of empathy. The saliency of all gestures was positively related to level of empathy. These results demonstrate that cognitive abilities and empathy levels are related to individual differences in gesture frequency and saliency.

The role of beat gesture and pitch accent in semantic processing: an ERP study.

The present study investigated whether and how beat gesture (small baton-like hand movements used to emphasize information in speech) influences semantic processing as well as its interaction with pitch accent during speech comprehension. Event-related potentials were recorded as participants watched videos of a person gesturing and speaking simultaneously. The critical words in the spoken sentences were accompanied by a beat gesture, a control hand movement, or no hand movement, and were expressed either with or without pitch accent. We found that both beat gesture and control hand movement induced smaller negativities in the N400 time window than when no hand movement was presented. The reduced N400s indicate that both beat gesture and control movement facilitated the semantic integration of the critical word into the sentence context. In addition, the words accompanied by beat gesture elicited smaller negativities in the N400 time window than those accompanied by control hand movement over right posterior electrodes, suggesting that beat gesture has a unique role for enhancing semantic processing during speech comprehension. Finally, no interaction was observed between beat gesture and pitch accent, indicating that they affect semantic processing independently.

The nature of gestures' beneficial role in spatial problem solving.

Co-thought gestures are hand movements produced in silent, noncommunicative, problem-solving situations. In the study, we investigated whether and how such gestures enhance performance in spatial visualization tasks such as a mental rotation task and a paper folding task. We found that participants gestured more often when they had difficulties solving mental rotation problems (Experiment 1). The gesture-encouraged group solved more mental rotation problems correctly than did the gesture-allowed and gesture-prohibited groups (Experiment 2). Gestures produced by the gesture-encouraged group enhanced performance in the very trials in which they were produced (Experiments 2 & 3). Furthermore, gesture frequency decreased as the participants in the gesture-encouraged group solved more problems (Experiments 2 & 3). In addition, the advantage of the gesture-encouraged group persisted into subsequent spatial visualization problems in which gesturing was prohibited: another mental rotation block (Experiment 2) and a newly introduced paper folding task (Experiment 3). The results indicate that when people have difficulty in solving spatial visualization problems, they spontaneously produce gestures to help them, and gestures can indeed improve performance. As they solve more problems, the spatial computation supported by gestures becomes internalized, and the gesture frequency decreases. The benefit of gestures persists even in subsequent spatial visualization problems in which gesture is prohibited. Moreover, the beneficial effect of gesturing can be generalized to a different spatial visualization task when two tasks require similar spatial transformation processes. We concluded that gestures enhance performance on spatial visualization tasks by improving the internal computation of spatial transformations. (PsycINFO Database Record (c) 2010 APA, all rights reserved).

Spontaneous gestures during mental rotation tasks: insights into the microdevelopment of the motor strategy.

This study investigated the motor strategy involved in mental rotation tasks by examining 2 types of spontaneous gestures (hand-object interaction gestures, representing the agentive hand action on an object, vs. object-movement gestures, representing the movement of an object by itself) and different types of verbal descriptions of rotation. Hand-object interaction gestures were produced earlier than object-movement gestures, the rate of both types of gestures decreased, and gestures became more distant from the stimulus object over trials (Experiments 1 and 3). Furthermore, in the first few trials, object-movement gestures increased, whereas hand-object interaction gestures decreased, and this change of motor strategies was also reflected in the type of verbal description of rotation in the concurrent speech (Experiment 2). This change of motor strategies was hampered when gestures were prohibited (Experiment 4). The authors concluded that the motor strategy becomes less dependent on agentive action on the object, and also becomes internalized over the course of the experiment, and that gesture facilitates the former process. When solving a problem regarding the physical world, adults go through developmental processes similar to internalization and symbolic distancing in young children, albeit within a much shorter time span..